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Background:  Human  patient  simula¬ 
tion  (HPS)  has  been  used  since  1969  for 
teaching  purposes.  Only  recently  has  tech¬ 
nology  advanced  to  allow  application  to 
the  complex  field  of  trauma  resuscitation. 
The  purpose  of  our  study  was  to  validate 
an  advanced  HPS  as  an  evaluation  tool  of 
trauma  team  resuscitation  skills. 

Methods:  The  pilot  study  evaluated 
10  three-person  military  resuscitation 
teams  from  community  hospitals  that  par¬ 
ticipated  in  a  28-day  rotation  at  a  civilian 
trauma  center.  Each  team  consisted  of 
physicians,  nurses,  and  medics.  Using  the 
HPS,  teams  were  evaluated  on  arrival  and 
again  on  completion  of  the  rotation.  In 
addition,  the  10  trauma  teams  were  com¬ 
pared  with  5  expert  teams  composed  of 
experienced  trauma  surgeons  and  nurses. 
Two  standardized  trauma  scenarios  were 
used,  representing  a  severely  injured  pa¬ 
tient  with  multiple  injuries  and  with  an 


Injury  Severity  Score  of  41  (probability  of 
survival,  50%).  Performance  was  mea¬ 
sured  using  a  unique  human  performance 
assessment  tool  that  included  five  scored 
and  eight  timed  tasks  generally  accepted 
as  critical  to  the  initial  assessment  and 
treatment  of  a  trauma  patient.  Scored 
tasks  included  airway,  breathing,  circula¬ 
tion,  and  disability  assessments  as  well  as 
overall  organizational  skills  and  a  total 
score.  The  nonparametric  Wilcoxon  test 
was  used  to  compare  the  military  teams’ 
scores  for  scenarios  1  and  2,  and  the  com¬ 
parison  of  the  military  teams’  final  scores 
with  the  expert  teams.  A  value  of  p  <  0.05 
was  considered  significant. 

Results:  The  10  military  teams  dem¬ 
onstrated  significant  improvement  in  four 
of  the  five  scored  (p  <  0.05)  and  six  of  the 
eight  timed  (p  <  0.05)  tasks  during  the 
final  scenario.  This  improvement  reflects 
the  teams’  cumulative  didactic  and  clini¬ 


cal  experience  during  the  28-day  trauma 
refresher  course  as  well  as  some  degree  of 
simulator  familiarization.  Improved  final 
scores  reflected  efficient  and  coordinated 
team  efforts.  The  military  teams’  initial 
scores  were  worse  than  the  expert  group 
in  all  categories,  but  their  final  scores 
were  only  lower  than  the  expert  groups  in 
2  of  13  measurements  (p  <  0.05). 

Conclusion:  No  studies  have  vali¬ 
dated  the  use  of  the  HPS  as  an  effective 
teaching  or  evaluation  tool  in  the  complex 
field  of  trauma  resuscitation.  These  pilot 
data  demonstrate  the  ability  to  evaluate 
trauma  team  performance  in  a  reproduc¬ 
ible  fashion.  In  addition,  we  were  able  to 
document  a  significant  improvement  in 
team  performance  after  a  28-day  trauma 
refresher  course,  with  scores  approaching 
those  of  the  expert  teams. 
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The  introduction  of  simulation  into  the  fields  of  medical 
training  and  competency  evaluation  is  a  relatively  new 
endeavor;  however,  the  effective  use  of  simulation  tech¬ 
nology  in  fields  such  as  aviation  training,  military  war  fighter 
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preparation,  and  industry  has  been  demonstrated  for  many 
years.1-6  Initial  research  outside  of  medicine  has  demon¬ 
strated  significant  overall  cost  savings  as  well  as  improved 
trainee  performance  with  the  integration  of  simulation  into 
their  training  matrix.12  With  recent  advances  in  computer 
technologies,  simulator-based  learning  and  evaluation  in  the 
medical  and  allied  health  communities  has  become  a 
possibility.7-10 

The  human  patient  simulator  (HPS)  was  first  introduced 
to  the  medical  community  in  1969  by  Denson  and  Abraha- 
mson,  who  used  a  patient  anesthesia  simulator  to  augment 
resident  training.5,6  Their  study  showed  a  trend  toward  faster 
skill  acquisition  and  subjective  performance  improvement. 
The  anesthesia  community  in  particular  has  embraced  the 
HPS  to  teach  technical  and  crisis  management  skills.  Over 
the  intervening  years,  the  medical  community  has  witnessed 
the  proliferation  of  personal  computer-based  teaching  mod¬ 
ules  and  medical  mannequins  that  are  becoming  more 
interactive.11-14  These  educational  devices  have  been  com¬ 
bined  in  the  advanced  HPS.  The  present  day  model  is  phys¬ 
iologically  based  and  reacts  to  interventions  in  real  time, 
which  forces  the  trainee  to  dynamically  interact  with  the 
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Trauma  Team  Performance  Measured  Using  a  Simulator 


device  and  proceed  through  the  complex  decision-making 
processes  involved  in  actual  patient  care. 

Earlier  simulation  studies  focused  on  individual  perfor¬ 
mance,  yet  clinicians  recognize  that  optimal  trauma  care  is 
delivered  by  experienced  multidisciplinary  teams. 15-18  Be¬ 
fore  the  widespread  acceptance  of  simulation  as  a  training 
platform,  we  believed  that  a  measurement  tool  of  human  team 
performance  should  be  developed.  This  tool  would  then  doc¬ 
ument  a  change  in  a  team’s  performance  after  an  educational 
intervention.  In  addition,  no  studies  have  validated  the  use  of 
the  HPS  as  an  effective  teaching  or  evaluation  tool  in  the 
complex  field  of  trauma  team  resuscitation.  We  report  herein 
the  results  of  the  repeated  use  of  the  HPS  coupled  with  a 
human  performance  evaluation  tool  over  the  course  of  a 
28-day  trauma  refresher  course. 

MATERIALS  AND  METHODS 

The  HPS  system  used  for  this  study  was  the  METI 
Advanced  Human  Patient  Simulator  (Medical  Education 
Technologies,  Inc.,  Sarasota,  FL).  The  space  for  the  simula¬ 
tion  center  was  a  converted  patient  care  area  located  in  the 
Ben  Taub  General  Hospital.  The  simulation  center  staff  con¬ 
sisted  of  active  duty  military  personnel  assigned  to  the  Joint 
Trauma  Training  Center  (JTTC).  These  personnel  were  either 
board-certified  surgeons  or  critical  care  physicians,  nurses,  or 
medics  with  extensive  trauma  experience  and  interest.  Each 
scenario  required  three  staff  members  to  monitor  the  com¬ 
puter/simulator,  manage  the  scenario,  and  videotape  the  ses¬ 
sions.  All  equipment  used  for  the  scenarios  was  standard 
medical  equipment  with  which  the  team  was  already  familiar. 

The  initial  process  of  developing  the  simulator  scenarios 
and  assessment  tool  involved  the  training  and  familiarization 
of  the  JTTC  staff  with  the  HPS  to  include  its  unique  potential 
applications  as  well  as  its  limitations.  The  staff  had  to  become 
facile  with  the  operation  of  the  simulator  to  provide  a  con¬ 
sistent  experience  for  the  rotators  in  addition  to  eliminating 
any  potential  inconsistencies  that  might  result  from  interin¬ 
structor  variability.  This  process  was  accomplished  by  an 
initial  week  of  hands-on  interactive  training  with  representa¬ 
tives  from  METI  during  the  installation  of  the  simulator.  This 
was  then  followed  by  several  months  of  practice  simulator 
sessions  and  scenario  development.  The  instructors  used  and 
refined  the  trauma  scenarios  as  well  as  the  assessment  tool  for 
several  months  before  initiating  the  formal  pilot  study. 

The  study  participants  were  members  of  active  duty 
military  trauma  teams  (Army,  Air  Force,  and  Navy,  16-24 
personnel)  performing  a  28-day  JTTC  rotation  at  the  Ben 
Taub  General  Hospital.  All  members  of  the  team  were  fully 
credentialed  in  their  respective  specialties.  The  majority  of 
the  teams  were  assigned  at  facilities  that  cared  for  few  acute 
trauma  victims.  The  physicians,  nurses,  nurse  anesthetists, 
and  medics  all  were  current  in  their  respective  Advanced 
Trauma  Life  Support  (ATLS),  Trauma  Nurse  Clinical  Core, 
or  Prehospital  Trauma  Life  Support  courses.  The  10  three- 
person  teams  underwent  a  brief  simulator  familiarization  ses- 
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Fig.  1.  The  simulator  being  used  by  a  three-person  military  team 
while  observed  by  an  experienced  trauma  physician. 


sion  and  then  were  evaluated  with  an  initial  simulator  sce¬ 
nario  (Fig.  1).  They  then  participated  in  an  intensive  28-day 
trauma  rotation  consisting  of  4  weeks  (72  hours  each)  of 
hands-on  clinical  experience,  trauma  case  reviews,  24  trauma 
lectures,  four  skill  station  sessions,  and  a  before  and  after  test 
of  didactic  trauma  knowledge.  At  the  conclusion  of  this 
trauma  immersion,  they  were  again  evaluated  in  the  simula¬ 
tion  center  on  the  final  scenario.  The  individual  composition 
of  the  teams  was  identical  at  both  time  points.  Two  standard¬ 
ized  and  physiologically  identical  trauma  scenarios  were 
used,  representing  a  severely  injured  patient  with  multiple 
injuries  and  with  an  Injury  Severity  Score  of  41,  (scenarios  1 
and  2,  both  with  significant  head,  chest,  abdomen,  and  ex¬ 
tremity  injuries;  probability  of  survival,  50%).  The  simulated 
patients’  underlying  physiology  and  required  interventions 
were  identical,  allowing  meaningful  comparisons  of  required 
scored  tasks  and  timed  interventions.  However,  the  scripted 
scenarios  presented  to  the  rotators  were  significantly  different 
to  avoid  pattern  recognition.  On  questioning  at  the  conclusion 
of  the  28-day  rotation,  the  teams  did  not  recognize  that  the 
scenarios  were  physiologically  identical.  The  scenarios  were 
also  constructed  in  a  fashion  that  would  allow  appropriate 
evaluation  and  disposition  within  a  15-minute  time  period. 
The  criteria  of  an  effective  resuscitation  were  defined  on  the 
basis  of  ATLS,  Prehospital  Trauma  Life  Support,  and  Trauma 
Nurse  Clinical  Core  guidelines,  with  a  distinct  military  em¬ 
phasis,  as  these  were  the  primary  focus  groups  for  the  JTTC 
rotators.19-21  As  there  are  no  consistent  nationally  accepted 
standards  for  team  evaluation  that  could  form  the  basis  of  our 
criteria,  we  applied  the  regionally  accepted  standards  of  care 
and  extrapolated  them  to  our  scenarios.  Once  these  criteria 
were  established,  an  evaluation  tool  was  created  to  capture 
the  data  and  score  the  performances.  This  tool  included  five 
scored  and  eight  timed  tasks,  all  generally  accepted  as  critical 
to  the  initial  assessment  and  treatment  of  a  trauma  patient. 
Scored  tasks  included  organizational  skills,  airway,  breath- 
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ing,  circulation,  disability,  and  total  score.  Timed  tasks  were 
standard  interventions  common  to  trauma  patients.  Perfor¬ 
mance  was  measured  through  videotape  review  of  the  ses¬ 
sions  using  the  human  performance  assessment  tool.  The 
videotapes  were  recorded  using  a  standard  digital  hand-held 
video  camera  with  built-in  sound  capabilities.  The  recorded 
tapes  were  subsequently  reviewed  and  scored  by  one  of  the 
authors  (J.W.C.).  The  current  version  of  the  assessment  tool 
used  in  this  work  evolved  over  12  months  of  use,  and  repre¬ 
sents  a  consensus  of  a  multidisciplinary  team  expert  in  trauma 
care  (Fig.  2).  The  evaluation  tool  was  designed  specifically 
for  our  scenarios,  and  an  emphasis  was  placed  on  objective 
measurements  to  improve  reliability.  The  assessment  tool 
should  be  considered  a  dynamic  document  that  can  be  mod¬ 
ified  to  fit  other  scenarios.  As  part  of  the  larger  JTTC  edu¬ 
cational  program,  identical  subjective  questions  were  asked 
of  the  trauma  teams  on  arrival  and  again  after  their  28-day 
trauma  experience. 

The  five  expert  teams  were  composed  of  experienced 
trauma  surgeons  and  shock  room  nurses  from  the  Ben  Taub 
General  Hospital.  The  expert  teams  provided  a  standard  per¬ 
formance  level  for  meaningful  comparison.  They  underwent 
the  same  brief  simulator  familiarization  as  the  rotating  mili¬ 
tary  teams  and  were  videotaped  and  scored  on  the  same  final 
scenario. 

The  nonparametric  Wilcoxon  test  was  used  to  compare 
the  military  teams’  scores  at  the  initial  and  final  assessments, 
and  the  comparison  of  the  military  teams’  final  scores  with 
the  expert  groups.  A  value  of  p  <  0.05  was  considered 
statistically  significant.  Statistical  analysis  was  performed 
using  the  SPSS  software  package  (Version  10.0,  SPSS,  Inc., 
Chicago,  IL). 

RESULTS 

The  study  teams  demonstrated  improvement  in  four  of 
the  five  scored  (p  <  0.05)  and  six  of  the  eight  timed  (p  < 
0.05)  tasks  during  the  final  scenario  (Figs.  3  and  4).  This  also 
resulted  in  a  significant  improvement  in  their  overall  score 
between  the  initial  and  final  evaluation  scenarios.  All  of  the 
timed  tasks  with  the  exception  of  time  to  recognize  a  pneu¬ 
mothorax  and  the  indication  for  intubation  were  improved. 
This  resulted  in  an  overall  shorter  time  in  recognizing  as  well 
as  treating  the  immediately  life-threatening  injuries.  The  only 
scored  task  not  improved  on  was  in  the  disability  section.  In 
addition,  there  was  a  more  rapid  disposition  of  the  simulated 
patient  in  the  final  scenario.  This  improvement  reflects  a 
combination  of  the  teams’  cumulative  didactic  and  clinical 
experience  during  the  28-day  trauma  rotation  as  well  as  some 
degree  of  simulator  familiarization.  The  improved  final 
scores  reflected  efficient  and  coordinated  team  efforts  as  well 
as  improved  verbal  and  nonverbal  communication.  In  re¬ 
sponse  to  the  subjective  question  described  previously,  the 
trauma  teams  felt  more  comfortable  caring  for  critically  ill 
trauma  patients  after  the  28-day  rotation  than  on  arrival. 


When  compared  with  the  expert  teams,  there  were  sig¬ 
nificant  differences  between  all  measured  and  timed  tasks  in 
the  initial  scenario.  However,  only  the  organizational  and 
recognition  of  the  need  for  a  laparotomy  scores  were  different 
(p  <  0.05)  between  the  teams’  final  scenario  and  the  experts’ 
scores.  Otherwise,  the  rotating  military  teams’  final  scores 
approached  the  expert  teams’  scores. 

DISCUSSION 

The  principal  finding  of  this  study  was  that  a  human 
performance  team  evaluation  tool  could  be  developed  to 
measure  the  ability  of  a  multidisciplinary  trauma  team  to 
diagnose  and  treat  a  “patient”  with  multiple  injuries.  We  then 
used  this  tool  to  show  the  improvement  in  performance  of  10 
teams  after  a  28-day  trauma  refresher  course.  The  initial 
challenges  of  this  study  included  familiarizing  the  staff  with 
the  capabilities  of  this  unique  teaching  tool  as  it  applies  to 
trauma  team  training  and  to  define  an  acceptable  set  of  team 
performance  standards  derived  from  existing  national  guide¬ 
lines.  The  guidelines  and  course  material  were  designed, 
however,  for  individual  instruction  and  assessment.  The 
guidelines  were  reviewed  and  the  critical  components  were 
combined  into  an  evaluation  tool  that  could  be  used  to  mea¬ 
sure  team  performance.  Each  scored  and  measured  task  was 
clearly  defined  so  as  to  provide  measurable  results.  One 
individual  performed  the  scoring  during  this  study  and  was 
intimately  involved  in  the  creation  and  refinement  of  the 
assessment  tool. 

To  our  knowledge,  such  a  trauma  team  assessment  tool 
has  not  been  used  for  serial  assessment  of  team  performance 
on  simulators  before  and  after  an  intense  educational  experi¬ 
ence.  The  process  and  tool  were  refined  for  several  months 
before  initiation  of  the  pilot  study.  This  trial  period  allowed 
us  to  become  familiar  with  the  use  of  the  HPS  and  to  calibrate 
the  evaluation  tool.  Use  of  the  videotape  review  was  essen¬ 
tial,  allowing  capture  of  the  dynamic  nature  of  the  trauma 
resuscitation.  Replay  of  the  tapes  was  not  only  critical  for 
scoring  the  teams  but  of  great  instructional  value  for  the 
teams.19  Training  of  the  videographer  was  imperative  to  cap¬ 
ture  the  dynamic  interaction  of  all  the  team  members  as  well 
as  the  physiologic  data  on  the  monitor,  which  allowed  for 
accurate  assessment  during  the  video  review.  The  time  spent 
learning  how  to  best  use  this  advanced  educational  technol¬ 
ogy  was  considered  critical  to  the  success  of  the  final  project. 

The  HPS  used  for  this  study  also  has  the  ability  to 
capture  the  variation  in  all  physiologic  data  for  the  entire 
duration  of  an  individual  scenario;  however,  this  capability 
was  not  used  for  this  study.  These  data  points,  such  as 
frequency  and  duration  of  systolic  blood  pressures  less  than 
90  mm  Hg  and  arterial  oxygen  saturations  less  than  90%,  may 
ultimately  decrease  the  need  for  subjective  assessments  of 
various  data  points  in  future  studies.  The  evaluation  and 
correlation  of  the  physiologic  data  as  they  pertain  to  overall 
team  performance  as  well  as  patient  outcome  is  ongoing. 
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Trauma  Team  Evaluation  Tool 


Team:  |Date;  |  Session#:  |start  Time: |  Finish  Time: 

Team  Leader:  (Circle  one) _ MD _ CRNA _ PA _ Med  Tech/Medic/Corpsman 

Total  Team  Members: 

Evaluator: 

Scenario  # 


1.  Team  Organization 

1 .  Clearly  defined  team  leader  emerges 

0=no  delegation/no  instruction 

2=delegates/instructs  others  throughout 

2.  Other  members  assume  functional  roles 

0=  uninvolved  personnel 

2=all  personnel  participate  actively 

3.  Verbal  communication  within  team 

0=no  clear  instructions 

1  “some  findings  called  out 

2=clear,  2-way  communication 

4.  Systematic  and  orderly  assessment 

(^disorganized 

1  =vertical  resuscitation 

2=horizontal  resuscitation 

5.  Ability  to  handle  distractions 

0=allow  attention  to  be  diverted 

1  temporary  diversion 

2= utilize  or  dismiss  distractor 

II.  Airway 

1 .  Airway  assessed 

0=no  assessment 

2=look/listen/feel,  talk  to  patient 

2.  Oxygen  applied 

0=  >  60  seconds 

1=  30-60  seconds 

2=  <  30  seconds 

3.  Recognize  indication  for  intubation 

0=  >  3  minutes 

1=  1:31  -3:00 

2=  <  1:30 

4.  Intubation 

a.  Appropriate  use  of  oxygen 

0=no  preoxygenation/bagging 

2=preoxygenates  and  btw.  Attempts 

b.  Correct  positioning 

0=no  c-spine  prec./position  with  blade 

2=c-spine  prec./sniffing  position 

c.  Correct  laryngoscopic  technique 

0=unfamiliarity  with  equipment 

2=checks  equipment/uses  cor 

d.  Endotracheal  tube  secured 

0=not  secured 

1  =afterthought/delayed 

2=secured  immediately 

e.  Reassessment  of  patient 

0=none 

1  =lungs/epigastrium  ausc. 

2=above  +  monitor  check/  ETC02  cons. 

5.  Recognize  indication  for  cricothyroidotomy 

0=  >  3  minutes 

1=1-3  minutes 

2=  <  1  minute 

6.  Cricothyroidotomy 

a.  Appropriate  use  of  oxygen 

0=no  bagging 

1=some  bagging 

2=throughout  attempt 

b.  Verbalized  correct  landmarks 

0=not  done/incorrect 

1=in  complete 

2=correct 

c.  Correct  insertion  technique 

0=no  dilation/opening  of  passage 

1  =above  done  but  mainstem  intubation 
2=above  +  correct  position 

d.  Endotracheal  tube  secured 

0=not  done/incorrect 

1  “delayed 

2=immediately 

e.  Reassessment  of  patient 

0=non@ 

1=lungs/epigastrium  ausc. 

2=above  +  monitor  check/ETC02  cons. 

7.  Time  to  secure  airway 

0=  >  5  minutes 

1=  3-5  minutes 

2=  <  3  minutes 

III.  Breathing 

1.  Breathing  assessed 

a.  Auscultation 

0=  >  60  seoonds 

1=  30-60  seconds 

2=  <  30  seconds 

2.  Recognized  tension  pneumothorax 

a.  Difference  in  auscultated  breath  sounds 
(time  to  awareness  of  difference) 

0=  >  3  minutes 

1=  1:30-3:00 

2=  <  1:30 

b.  Time  to  decompression  of  ptx 

0=  >  4  minutes 

1=  2-4  minutes 

2=  <  2  minutes 

3.  Needle  thoracentesis 

a.  Verbalized  landmarks 

0=incorrect 

1= in  complete 

2= correct 

b.  Verbalized  proper  technique 

0=incorrect 

1  “incomplete 

2=over  rib/midciavicular  line 

c.  Reassessment  of  patient 

1  =breath  sounds 

2  “breath  sounds  +  vitals 

Fig.  2.  The  trauma  team  evaluation  tool. 


4.  Recognized  indication  for  tube  thoracostomy 

a.  Verbalized  equipment  required 
(tube,  scalpel,  Kelly,  suture,  drainage) 

0=unsure  of  equipment 

1  incomplete 

2=all  equipment  identified 

b.  Verbalized  landmarks 

0=incorrect 

1  “incomplete 

2=reasonable 

c.  Verbalized  insertion  technique 

0=incorrect 

1  “incomplete 

2=over  rib,  blunt  dissect,  digital  insertion 

d.  Secured  tube 

0=not  done 

1  “delayed  or  incorrect 

2=done  immediately 

e.  Attached  tube  to  pleur-evac  system 

0=not  done 

1  “incomplete 

2=complete/suction  on 

IV.  Circulation 

1 .  Checked  monitor  for  BP,  HR 

0=  >  1  minute 

(time  to  recognition  of  vitals) 

2=  <  30  seconds 

2.  Assessed  pulses 

0=not  checked 

1 “checked 

2=noted  absent  RLE  pulse 

3.  Applied  pressure  to  stop  bleeding 

0=  >  1  minute 

(time  to  adequate  pressure  applied) 

1  =  30  -  60  seconds 

2=  <  30  seconds 

4.  Applied  tourniquet  appropriately 

0=  >  3  minutes 

(time  to  tourniquet  application  complete) 

1=1-3  minutes 

2=  <  1  minute 

5.  Tourniquet  application  technique 

0=incorrect  location  or  loose 

1  “did  not  reassess 

2=proper  application  and  reassessment 

6.  Time  to  IV  access  achieved 

0=  >  3  minutes 

1=  1:30-3:00 

2=  <  1:30 

V.  Disability 

1 .  Assessed  responsiveness 

0=no 

1  “incomplete 

2=yes 

2.  Pupils 

0=not  examined 

1  = 

2=examined 

3.  Aware  of  neurologic  injury 

0=not  aware 

1  “asymmetry  noted;  unaware  of  sign. 
2=siated  neurologic  injury 

4.  Verbalized  therapies  for  closed  head  injury 

0=none 

(hyperventilate,  higher  MAP,  drugs,  crani) 

1=1-2  therapies 

2=>2  therapies  offered 

5.  Examination  of  abdomen 

0=not  examined 

1= 

2=examined 

6.  Recognized  indication  for  laparotomy 

0=not  recognized 

1= 

2=stated  lap  indicated 

7.  Time  to  recognize  lap  indication 

0=  >  1 3  min 

1=  10-13  minutes 

2=  <  10  minules 

8.  Full/complete  exposure 

0=none 

1  “incomplete 

2=complete 

9.  C-spine  precautions 

0=none 

1  “sometimes 

2=throughout 

VI.  Timed  tasks 

1 .  Time  to  recognition  of  vitals 

2.  Time  to  oxygen  applied 

3.  Time  to  adequate  pressure  applied  to  extremity 

4.  Time  to  auscultation 

5.  Time  to  recognition  of  pneumothorax 

6.  Time  to  tourniquet  applied 

7.  Time  to  chest  decompression 

8.  Time  to  recognition  of  indication  for  intubation 

9.  Time  to  recognition  of  indication  for  cricothyroidotomy 

(from  first  intubation  attempt) 

1 0.  Time  to  secure  airway 

1 1 .  Time  to  IV  access  obtained 

12,  Time  to  recognize  laparotomy  indication 

VII.  Total  score _ 

1 .  Total  points 

2.  Total  points  possible  [ 

3.  Percentage [ 


The  HPS  was  used  as  only  one  component  of  the  larger 
JTTC  educational  effort.  The  educational  design  of  the  pro¬ 
gram  consisted  of  five  distinct  areas:  hands-on  experience, 
comprehensive  trauma  lectures  and  skills  stations,  before  and 


after  testing,  subjective  questions  and  follow-up  surveys, 
and  the  trauma  simulation  center.  Although  the  28-day 
rotation  was  primarily  a  hands-on  learning  experience  for 
a  multidisciplinary  trauma  team,  the  educational  design 
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Scored  Data 


90.00 


s  60.00 
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Fig.  3.  The  percentage  of  the  five  scored  tasks  and  the  total  score  for  the  military  and  expert  teams.  *p  <  0.05  between  scenario  1  and 
scenario  2.  **p  <  0.05  between  scenario  2  and  expert  teams. 


included  self-confidence  type  questions  focused  on  the 
individual  and  team  aspects  of  their  learning  experience. 
There  was  not  a  specific  question  directed  toward  assess¬ 
ing  the  self-confidence  effect  of  the  simulator,  but  rather 


the  effect  of  the  entire  28-day  rotation.  In  response  to  the 
subjective  questions,  the  trauma  teams  felt  more  comfort¬ 
able  caring  for  critically  injured  trauma  patients  after  the 
28-day  rotation  than  on  arrival.  In  addition,  the  rotators 


Timed  Intervention  Data 


□  Scenario  1  * 

□  Scenario  2 

■  Expert  Group 

T  *  T 

** 

* 

* 

Ail 

♦ 

fei  1 

[L 

il 

Oxygen  applied  Auscultation  PTX  recognition  Tourniquet  Chest  Recognize  Secure  airway  Recognize 

decompression  indication  for  laparotomy 

intubation  indication 


Intervention 

Fig.  4.  Differences  in  minutes  between  scenarios  1  and  2  for  the  military  teams  and  expert  teams  for  the  eight  timed  interventions.  *p  <  0.05 
between  scenario  1  and  scenario  2.  **p  <  0.05  between  scenario  2  and  expert  teams. 
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universally  wanted  to  spend  more  time  on  the  simulator 
than  was  allocated. 

Many  studies  using  simulation  technology  have  demon¬ 
strated  subjective  performance  improvement  as  well  as  in¬ 
creased  subjective  comfort  levels  with  individual  tasks;  how¬ 
ever,  no  studies  have  objectively  quantified  the  performance 
of  a  team  during  the  complex  and  dynamic  interactions  in¬ 
volved  in  trauma  resuscitation.  Simulation  studies  on  aircraft 
crew  performance  demonstrate  increased  comfort  with  in¬ 
flight  emergencies  as  well  as  decreased  overall  training  cost 
when  simulation  technology  is  added  to  the  training 
matrix.1-3  It  is  reasonable  to  assume  that  this  type  of  training 
may  also  benefit  those  in  the  medical  field.  However,  to 
demonstrate  an  improvement  using  simulation  technology,  a 
valid  and  reproducible  assessment  modality  must  be  created. 
Our  data  demonstrate  the  ability  to  capture  and  quantitatively 
assess  team  performance  and  measure  improvement  after  the 
JTTC  trauma  experience  using  the  HPS  in  a  controlled  envi¬ 
ronment  (Figs.  3  and  4). 

In  our  preliminary  work  (not  reported)  with  the  HPS,  two 
physiologically  different  scenarios  were  used.  The  day-1 
evaluation  was  a  relatively  simple  patient  problem,  whereas 
the  day-28  scenario  was  much  more  complex.  However,  to 
improve  our  study  design,  we  changed  the  scenarios  to  be 
physiologically  identical,  thus  requiring  the  same  interven¬ 
tions,  allowing  more  meaningful  comparisons  of  times,  mul¬ 
tiple  complex  interventions,  team  interactions,  and  the  effect 
of  the  28-day  trauma  rotation.  The  scripted  scenarios  pre¬ 
sented  to  the  rotators  were  significantly  different  to  avoid 
pattern  recognition.  As  part  of  the  larger  JTTC  rotation, 
significant  subjective  data  were  recorded  and  universally  the 
teams  did  not  recognize  that  the  day-1  and  day-28  scenarios 
were  essentially  the  same. 

The  ability  to  transfer  the  complex  skill  of  trauma  eval¬ 
uation  and  resuscitation  effectively  to  the  clinical  environ¬ 
ment  as  well  as  determining  skill  degradation  rates  and  train¬ 
ing  frequencies  still  require  further  investigation.1'22-24 
Additional  questions  include  the  following:  How  “much” 
trauma  does  a  team  need  to  manage  to  maintain  their  evalu¬ 
ation  skills  once  they  are  functioning  at  a  high  level?  How 
fast  can  they  regain  those  degraded  skills  after  a  period  of 
clinical  inactivity?  The  HPS  technology  when  combined  with 
clinical  trauma  activity  and  a  focus  on  education  and  simu¬ 
lation  may  start  to  answer  these  questions  using  the  methods 
presented  herein. 

Although  this  study  only  focused  on  trained  military 
personnel  undergoing  a  trauma  refresher,  a  major  area  of 
obvious  benefit  of  the  HPS  is  in  teaching  high-risk  and/or 
low-incidence  clinical  scenarios  to  trainees  of  all  levels  and 
disciplines.  The  time-honored  tradition  of  the  student-teacher 
interaction  at  the  patient’s  bedside  is  still  the  teaching  mo¬ 
dality  of  choice;  however,  the  addition  of  the  HPS  may 
ultimately  prepare  the  students  for  more  efficient  learning  at 
the  bedside  and  maximize  the  time  spent  with  mentors.25-29 
This  potential  benefit  merits  further  investigation  as  to  the 


possibility  of  integration  of  this  unique  teaching  modality  into 
the  training  matrices  of  several  health  care  professions.  As  an 
example,  the  ATLS  course  has  recently  been  conducted  using 
simulation  devices  rather  than  animals,  with  approval  by  the 
American  College  of  Surgeons  Committee  on  Trauma  (Kauf- 
mann  et  ah,  unpublished  data).  Furthermore,  the  HPS  is  an 
excellent  crisis  management  tool.8  It  allows  the  physician, 
nurse,  and/or  medic  the  opportunity  to  practice  high-risk 
skills  in  a  risk-free  environment.  Their  “suspension  of  disbe¬ 
lief’  is  real,  as  these  trainees  feel  as  if  they  are  taking  care  of 
a  real  patient,  and  they  demonstrate  real  stress  when  the 
simulated  patient  does  not  do  well.  Fortunately,  when  this 
occurs,  the  scenario  can  be  restarted  and  replayed  until  the 
individual  or  team  “gets  it  right.”  From  an  adult  education 
point  of  view,  the  simulator  sessions  were  extremely  popular 
with  the  trauma  teams,  and  the  consensus  was  that  increased 
time  spent  in  the  simulation  center  would  be  beneficial. 

The  addition  of  the  expert  team  as  a  control  group  de¬ 
fined  an  important  standard  to  which  we  could  compare  the 
performance  of  the  rotating  trauma  teams,  and  demonstrated 
that  the  scoring  standards  were  realistic.  The  difference  be¬ 
tween  the  military  teams’  initial  scores  and  the  expert  team 
performance  was  significant  for  all  measured  variables.  How¬ 
ever,  the  military  teams’  final  scores  were  only  different  from 
the  experts  in  the  recognition  for  need  of  laparotomy  and  the 
overall  organizational  category.  All  other  scores  approached 
those  of  the  expert  group.  To  fully  define  this  component  of 
performance  enhancement,  we  need  to  allow  the  expert  team 
to  complete  a  second  scenario  at  a  28-day  interval  to  control 
for  any  degree  of  improvement  resulting  from  simulator  fa¬ 
miliarization.  This  component  will  be  added  to  future  studies. 

The  final  scores  of  the  military  teams  approached  those 
of  the  expert  teams.  This  may  have  occurred  for  several 
reasons.  The  intensive  experience  and  teaching  over  the  28- 
day  rotation  may  have  been  successful  in  bringing  the  trained 
surgeons,  nurses,  and  medics  up  to  an  expert  level.  Alterna¬ 
tively,  since  the  military  teams  had  additional  HPS  exposure, 
their  improved  scores  could  be  explained  by  a  greater  famil¬ 
iarity  with  the  simulator.  Finally,  perhaps  the  expert  teams 
were  not  quite  as  skilled  as  one  might  expect,  or  they  had 
difficulty  using  the  simulator.  In  fact,  the  expert  teams  did  not 
voice  any  overt  difficulty  with  the  simulator,  and  the  subjec¬ 
tive  assessment  by  the  simulation  center  staff  was  that  the 
performance  of  the  expert  teams  was  excellent.  We  believe 
that  the  improvement  seen  in  the  military  teams  is  largely 
because  of  a  real  improvement  in  the  team  resuscitation 
skills,  with  some  element  of  increased  simulator  familiarity. 

The  principal  strength  of  the  study  is  that  the  rotating 
military  teams  were  all  fully  trained  and  certified  in  their 
respective  areas  of  expertise,  and  experienced  a  uniform 
clinical  and  didactic  trauma  rotation.  On  entry  into  the  study, 
few  had  any  recent  trauma  experience.  The  difference  be¬ 
tween  the  initial  snapshot  of  their  ability  to  diagnose  and 
manage  a  difficult  trauma  simulation  and  their  score  28  days 
later  reflects  the  benefit  of  undergoing  an  intense  trauma 
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refresher  experience.  From  a  military  point  of  view,  the 
reproducible  and  documented  improvement  in  evaluation  and 
decision-making  skill  of  the  trauma  team  is  important  infor¬ 
mation  in  the  ongoing  debate  about  the  usefulness  of  a  re¬ 
curring  exposure  to  trauma  during  one’s  military  career. 

There  are  several  weakness  in  this  study.  The  principal 
problem  is  that  we  did  not  control  for  simulator  familiarity 
between  the  military  and  expert  groups.  This  could  have  been 
controlled  by  retesting  the  expert  groups  28  days  after  their 
initial  evaluation,  with  any  improvement  attributed  to  simu¬ 
lator  familiarity  rather  than  improvement  in  clinical  skill. 
Although  initially  planned,  unfortunately  this  did  not  occur. 
The  measurement  tool  was  not  validated  in  a  standard  edu¬ 
cational  fashion  with  techniques  such  as  repeated  observa¬ 
tions  in  large  groups,  naive  evaluators,  and  rigorous  evalua¬ 
tion  of  interobserver  reliability.  However,  with  the  assistance 
of  a  trained  educator  (C.E.S.),  the  tool  went  through  four 
versions  and  in  the  form  presented  did  document  the  expected 
increase  in  clinical  performance  after  an  intensive  28-day 
trauma  rotation.  In  addition,  the  expert  group  scored  better 
than  the  rotators,  adding  further  validity  to  the  measurement 
tool.  This  initial  study  was  designed  to  help  form  the  back¬ 
ground  for  further  work,  with  the  full  expectation  that  the 
human  performance  evaluation  tool  will  be  modified  (Fig.  2). 
Furthermore,  the  evaluation  tool  has  not  been  validated  in 
real  trauma  resuscitations.  The  assessment  tool  should  be 
used  in  a  clinical  environment  that  currently  uses  video  as¬ 
sessment  of  ongoing  trauma  resuscitations.  Finally,  and  most 
important,  we  do  not  know  whether  the  improvement  in 
performance  documented  by  the  simulator  will  transfer  to  the 
real  world  of  trauma  resuscitation  at  2:00  am.  The  airline 
industry,  space  program  ,and  the  U.S.  military  currently  use 
simulation  training  extensively.  Each  of  these  organizations 
uses  simulation  for  initial  and  sustainment  training,  and  they 
also  practice  crisis  management  skills  on  a  routine  basis. 
Trauma  resuscitations  on  critically  injured  patients  are  in 
effect  a  controlled  crisis  management  activity,  albeit  one 
practiced  on  a  routine  basis  at  busy  centers.  It  is  a  learned 
skill  that  improves  over  time,  and  the  HPS  may  decrease  that 
learning  time.  If  one  believes  the  simulation  data  from  the 
airline,  space,  and  military  environments,  then  transference 
from  the  HPS  to  the  clinical  situation  should  occur.  This 
remains  to  be  definitively  demonstrated. 

The  concepts  of  virtual  reality  and  simulation  are  gaining 
increasing  acceptance  in  many  fields  of  medical  education 
and  training  to  include  the  surgical  disciplines.  ’  Al¬ 
though  the  HPS  systems  currently  do  not  integrate  virtual 
reality  components,  efforts  are  underway  to  merge  these 
technologies.  This  is  a  desirable  direction  because,  once  this 
occurs,  actual  surgical  procedures  can  be  incorporated  into 
the  training  scenarios,  allowing  a  complete  continuum  of 
trauma  care  on  the  simulator. 

Although  not  evaluated  in  the  current  study,  we  also  used 
the  HPS  for  evaluation  and  familiarization  with  new  devices 
and  for  teaching  purposes.  The  videotapes  that  were  used  for 


Fig.  5.  Using  the  HPS  to  evaluate  new  portable  anesthesia 
equipment. 


scoring  purposes  were  also  reviewed  with  each  military  team 
for  their  educational  value.  In  addition,  150  medical  students 
were  introduced  to  the  basic  concepts  of  trauma  patient  eval¬ 
uation  using  the  HPS.  As  the  HPS  used  in  this  study  simulates 
the  gas  exchange  of  a  ventilated  human,  we  used  this  feature 
to  evaluate  a  new  portable  anesthesia  device  (Fig.  5).  Allow¬ 
ing  the  rotating  military  personnel  to  become  familiar  with 
the  device  on  the  simulator  rather  than  on  a  patient  was  seen 
as  a  real  benefit.  The  HPS  is  an  excellent  device  for  these 
uses,  and  in  postrotation  comments  the  simulator  sessions 
were  counted  as  a  highlight  of  the  trauma  rotation.  With  the 
recent  heightened  public  concern  of  preventable  medical  er¬ 
rors,  efforts  to  institute  measures  that  avert  or  significantly 
reduce  problems  could  be  rigorously  evaluated  on  the  HPS 
platform.33  The  HPS  could  be  used  in  a  continuous  quality 
improvement  process  to  reproduce  identified  clinical  prob¬ 
lems,  and  then  train  individuals  or  teams  in  optimal  solutions. 

CONCLUSION 

These  data  demonstrate  the  ability  to  evaluate  trauma 
team  performance  in  a  reproducible  fashion.  This  initial  val¬ 
idation  step,  although  tedious  and  time  consuming,  is  a  nec¬ 
essary  step  before  any  institution  of  policies  that  assess  com¬ 
petencies  using  the  HPS.  Although  these  practices  are  already 
surfacing  in  various  health  care  fields,  the  first  step  is  to 
prove  that  the  HPS  can  measure  the  skill  in  question  and  that 
competency  (or  lack  of  competency)  on  the  simulator  corre¬ 
lates  with  the  presence  or  lack  of  clinical  competency.  The 
intense  28-day  trauma  experience  at  the  JTTC  resulted  in  a 
measurable  improvement  in  team  performance  in  the  pilot 
study  group.  Further  studies  using  larger  groups  and  multiple 
independent  reviewers  are  warranted  on  the  basis  of  these 
initial  pilot  data.  Future  studies  should  control  for  the  simu¬ 
lator  familiarization  effect.  Furthermore,  adding  automati¬ 
cally  recorded  physiologic  data  to  the  evaluation  process  will 
increase  the  collection  of  objective  data,  allowing  improved 
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discrimination  between  teams  and  possibly  decreasing  the 
tedious  process  of  videotape  review  required  to  generate  the 
team  scores. 

Ultimately,  the  initial  practice  and  acquisition  of  medical 
skills  (technical  and  evaluation)  may  take  place  in  the  simu¬ 
lation  environment.  For  military  and  civilian  prehospital  per¬ 
sonnel,  this  environment  could  be  modified  to  include  vari¬ 
ations  in  light,  sound,  available  assets,  and  numbers  of 
casualties.  This  risk-free  and  reproducible  environment  may 
be  an  ideal  modality  for  augmenting  the  training  of  individual 
medical  practitioners  as  well  as  multidisciplinary  teams. 
Linking  simulators  together  to  replicate  a  mass  casualty  event 
could  evaluate  not  only  individual  triage  decisions  but  also 
the  decisions  of  larger  teams  and  commanders.  In  no  way  do 
we  suggest  that  the  simulator  can  take  the  place  of  actual 
supervised  or  independent  clinical  experience.  Its  use  may, 
however,  better  prepare  those  who  are  about  to  venture  into 
the  clinical  arena  for  the  first  time  or  possibly  refresh  skills 
and/or  decision-making  processes  for  uncommon  or  infre¬ 
quent  occurrences. 
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EDITORIAL  COMMENT 

The  Human  Patient  Simulator  (HPS)  has  gained  increasing 
acceptance  as  a  tool  in  medical  education,  as  it  poses  no  clinical 
risks  to  patients  during  the  learning  process,  allows  for  simula¬ 
tion  and  thus  exposure  to  a  variety  of  potential  clinical  situations 
in  a  short  time  frame,  and  provides  for  team-building,  which  is 
an  essential  skill  in  complex  patient  care  management  scenarios 
including  trauma  resuscitation.  HPS  can  evaluate  decision-mak¬ 
ing,  assessment  skills,  diagnostic  skills,  and  timeliness  of  inter- 
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ventions  and  team  function.  These  are  useful  measures  for  some 
learning  situations,  particularly  the  one  described  in  this  paper. 
A  disadvantage  of  HPS  at  the  present  time  is  the  significant  fixed 
cost  associated  with  acquisition  and  the  ongoing  maintenance 
costs  for  the  equipment. 

The  authors  used  the  HPS  in  a  “before  and  after”  role 
with  a  28-day  trauma  rotation  as  the  experience  tool  in  be¬ 
tween.  Predictably,  performance  improved  after  the  trauma 
rotation  as  measured  by  team  performance  on  the  HPS.  Fur¬ 
ther  work  by  the  authors  and  others  will  be  required  to 


determine  the  long-term  effectiveness  of  the  HPS  in  training, 
but  as  an  initial  training  tool,  it  clearly  is  effective.  The  HPS 
may  be  an  important  adjunct  to  the  many  other  didactic 
trauma  training  programs  currently  in  use  and  may  replace 
“live”  simulation  laboratories  with  better  scenario-based 
learning  experiences. 
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